Python: Fix spurious runtime tools warning when using use_latest_version#4691
Python: Fix spurious runtime tools warning when using use_latest_version#4691jgarrison929 wants to merge 5 commits intomicrosoft:mainfrom
Conversation
Python Test Coverage Report •
Python Unit Test Overview
|
||||||||||||||||||||||||||||||
There was a problem hiding this comment.
Pull request overview
This PR fixes a false-positive warning emitted by AzureAIClient when use_latest_version=True and the framework supplies an empty runtime tools list ([]), reducing log noise and user confusion.
Changes:
- Adjusted
_remove_agent_level_run_options()to treat empty runtime tools as “no runtime tools supplied” when there is no creation-time baseline. - Updated the warning string formatting (same message content, reflowed).
- Added a regression test covering the
use_latest_version=True+ empty tools scenario.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
python/packages/azure-ai/agent_framework_azure_ai/_client.py |
Refines runtime tools/structured_output change detection to avoid warning on default empty tools without a baseline. |
python/packages/azure-ai/tests/test_azure_ai_client.py |
Adds regression test intended to ensure no warning is emitted for use_latest_version=True when tools are []. |
You can also share your feedback on Copilot code review. Take the survey.
giles17
left a comment
There was a problem hiding this comment.
Automated Code Review
Reviewers: 4 | Confidence: 92%
✓ Correctness
The change fixes a false-positive warning when
use_latest_version=Trueand the framework passes an empty runtime tools list. Previously, whenwarn_runtime_tools_and_structure_changedwas False, any non-Noneruntime_tools(including[]) would trigger a spurious mismatch warning. The new code introduces anelsebranch that uses a truthiness check (if runtime_tools:) so empty lists are ignored, while non-empty tool lists still correctly trigger the warning. The structured_output check correctly remainsis not Nonesince there is no analogous empty-default for that field. The test properly exercises the regression scenario withuse_latest_version=Trueand empty tools. Existing tests for non-empty tools withwarn_runtime_tools_and_structure_changed=False(e.g.,test_prepare_options_logs_warning_for_tools_with_existing_agent_version) are unaffected because non-empty lists are still truthy.
✓ Security Reliability
This diff fixes a false-positive warning when
use_latest_version=Truecauses the framework to pass an empty tools list ([]). Previously, theelsebranch (whenwarn_runtime_tools_and_structure_changedis False) unconditionally settools_changed = Truewheneverruntime_tools is not None, which includes the empty-list case. The fix correctly initializes both flags toFalseand, in theelsebranch, uses a truthiness check (if runtime_tools:) to skip empty lists. The logic is correct and the test adequately covers the regression. No security, reliability, or resource-leak concerns were identified.
✗ Test Coverage
The diff fixes a false-positive warning when
use_latest_version=Trueby distinguishing empty runtime tools ([]) from actual user-supplied tools. The new test (test_use_latest_version_no_spurious_warning_for_empty_tools) correctly exercises theelsebranch of_remove_agent_level_run_optionswith an empty tools list and asserts that no warning is emitted. However, there is no complementary test verifying that non-empty tools withuse_latest_version=Truestill correctly trigger the warning — this is the positive case for the sameelsebranch. Without it, a future regression that accidentally suppresses all warnings in theelsebranch would go undetected. Additionally, theruntime_structured_outputpath in theelsebranch remains untested whenwarn_runtime_tools_and_structure_changedis False.
✓ Design Approach
The fix correctly addresses a real false-positive warning: when
use_latest_version=True(or an explicitagent_versionis supplied),warn_runtime_tools_and_structure_changedstaysFalsebecause the client never calledcreate_version. Under the old code, an empty[]tools list passed by the framework was enough to settools_changed = True(because[] is not None), firing a spurious warning. The new approach initializes both flags toFalseand uses truthiness (if runtime_tools:) in the no-baseline branch to ignore an empty list, which is the right semantic. The_get_structured_output_signaturehelper already normalisesresponse_formatso it returnsNonefor missing values, meaningif runtime_structured_output is not Nonein the else-branch is reliable. The regression test correctly exercises theuse_latest_versionpath through_prepare_optionsend-to-end (not just the helper), which gives confidence the fix holds in the real call stack. One minor inaccuracy in the new comment: whenuse_latest_version=Truebut the agent does not yet exist, the fallback creation path setswarn_runtime_tools_and_structure_changed = True, so that case does NOT end up in theelsebranch — the comment implies it always does. This is a docs-only concern and does not affect correctness.
Flagged Issues
- Missing complementary test: there is no test verifying that non-empty tools with
use_latest_version=True(or explicitagent_version) still trigger the warning via the newelsebranch. The existingtest_prepare_options_logs_warning_for_tools_with_existing_agent_versioncoversagent_versionbut was written against the old logic. A dedicated test foruse_latest_version=Truewith non-empty tools is needed to guard against regressions that silently suppress all warnings in this code path.
Suggestions
- Add a test for
runtime_structured_output is not Nonewhenwarn_runtime_tools_and_structure_changed=Falseto cover the structured-output half of the newelsebranch. - Tighten the inline comment on the
elsebranch: whenuse_latest_version=Truebut the agent does not yet exist, the fallback creation path setswarn_runtime_tools_and_structure_changed = True, so this branch is never reached in that case. Consider wording like: 'Agent was fetched externally (use_latest_version found an existing agent, or explicit agent_version was supplied). We have no creation-time baseline…'
Automated review by giles17's agents
|
Ran prek hooks and pushed fixes per DEV_SETUP.md guidelines. Thanks @moonbox3! |
When use_latest_version=True, AzureAIClient fetches the existing agent from the service rather than creating one. In this flow the client never records creation-time tool names, so the subsequent _remove_agent_level_run_options check incorrectly treats an empty runtime tool list as a mismatch and always emits: 'AzureAIClient does not support runtime tools or structured_output overrides after agent creation.' Root cause: the 'tools_changed' flag was set to 'runtime_tools is not None', which evaluates to True even for an empty list (the framework default). When warn_runtime_tools_and_structure_changed is False we have no creation-time baseline to compare against. Fix: when no creation-time baseline exists, only flag 'tools_changed' when runtime_tools is *non-empty* (i.e. the user actually supplied tools). An empty tool list from framework defaults no longer triggers a false-positive warning. The existing behavior for agents created by the client (where we have a proper baseline to compare tool sets) is unchanged. Added a regression test that verifies no warning fires for the use_latest_version + empty tools path. Fixes microsoft#4681
…s calls Moved the logger.warning patch to wrap both the first and second _prepare_options calls, ensuring neither emits a spurious warning when use_latest_version=True with empty tools.
Companion test requested by giles17 — verifies that non-empty runtime tools still correctly trigger the mismatch warning when use_latest_version=True, guarding against regressions that suppress all warnings in the else branch.
…ename The upstream codebase renamed RawOpenAIResponsesClient to RawOpenAIChatClient and moved it from agent_framework.openai._responses_client to agent_framework_openai._chat_client. Update the two regression tests added in this PR to use the correct mock target path, matching the pattern used by all other tests in the file. Fixes CI Python test failures on all matrix entries.
17ad30c to
8dd81c4
Compare
|
Updated the two regression tests to use the correct mock path after the upstream |
|
Hey @jgarrison929 can you resolve the merge conflicts? |
|
AzureAIClient was removed, and the replacement FoundryAgent does not do any runtime tools to the service. So I will close this |
Motivation and Context
When using
AzureAIClientwithuse_latest_version=True, the SDK always emits:This warning fires on every request even when the user has not supplied any runtime tools, making it a false positive that creates noise in logs and confuses users.
Fixes #4681
Description
Root cause:
_remove_agent_level_run_options()setstools_changed = runtime_tools is not None, which evaluates toTrueeven for an empty list[]— the default from the framework's tool preparation machinery. Whenuse_latest_version=True, the client fetches an existing agent from the service rather than creating one, sowarn_runtime_tools_and_structure_changedis never set toTrueand the tool-set comparison branch that would catch "same tools" is never reached.Fix: When no creation-time baseline exists (
warn_runtime_tools_and_structure_changed is False), only flagtools_changedwhenruntime_toolsis non-empty (i.e., the user actually supplied tools). An empty tool list from framework defaults no longer triggers a false-positive warning.The existing behavior for agents created by the client (where we have a proper baseline to compare tool sets by name) is unchanged. The warning still fires correctly when:
agent_versionis set and non-empty tools are passedChanges:
python/packages/azure-ai/agent_framework_azure_ai/_client.py: Restructured thetools_changed/structured_output_changedlogic in_remove_agent_level_run_options()to distinguish between the "has baseline" and "no baseline" casespython/packages/azure-ai/tests/test_azure_ai_client.py: Added regression testtest_use_latest_version_no_spurious_warning_for_empty_toolsverifying no warning fires for theuse_latest_version+ empty tools pathContribution Checklist